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Abstract 

In this work, we extend the idea of Quantum Markov chains [S. Gud- 
der. Quantum Markov chains. J. Math. Phys., 49(7), 2008] in order to 
propose Quantum Hidden Markov Models (QHMMs). For that, we use 
the notions of Transition Operation Matrices (TOM) and Vector States, 
which are an extension of classical stochastic matrices and probability dis¬ 
tributions. Our main result is the Mealy QHMM formulation and proofs 
of algorithms needed for application of this model: Forward for general 
case and Vitterbi for a restricted class of QHMMs. We show the relations 
of the proposed model to other quantum HMM propositions and present 
an example application. 

keywords: Hidden Markov Models open quantum walks Transition 
Operation Matrices 


1 Introduction 

The most basic Markov model is a Markov chain, which can be defined as a 
stochastic process with the Markov property. Formally, a Markov chain is a 
collection of random variables {nt,t > 0} having the property that P(n,t+i = 
Sk t+1 \ni = S kl ,n 2 = S k2 ,...,n t = S kt ) = P(n t +i = S kt+1 \n t = S kt ), where the 
values {Si,S t} of n t are called states. They form the state space of the 
chain. According to the Markov property, the current state of a chain is only 
dependent on the previous state. Moreover, the state of a Markov chain is 
directly observed in each step. Any Markov chain can be described by a directed 
graph called the state diagram, where vertices are associated with states and 


1 


each edge (i,j) is labelled by the probability of going from i-tli state to j- th 
state. The information about Markov chain can be also represented by the 
initial state II and the stochastic matrix called transition matrix P = \ptj\, such 
that Pij = P(nt +1 = Sj\nt = Si). If we consider a Markov chain where states 
are not observed directly, and these states generate symbols according to some 
random variables, then we obtain a Hidden Markov Model (HMM). Hence, in 
the case of a Markov chain, the states correspond with observations, but for a 
HMM, the states correspond with the random source of observations. 

The classical Hidden Markov Model was introduced as a method of modelling 
signal sources observed in noise. It is now extensively used, e.g. in speech 
and gesture recognition or biological sequence analysis. Their popularity is 
a result of their versatile structure, which is able to model wide variety of 
problems, and effective algorithms that facilitate their application. The HMM 
is related to three fundamental: given a sequence of symbols of length T, O = 
(oi, 02 ,..., or), and a HMM parametrized by A, 

1. Compute the P(0|A), probability that the sequence O can be produced 
by a HMM A. 

2. Select the sequence of state indexes Nt = (n o, ni,. .., tit) that maximizes 
the probability P(0|A, Nt); in other words the most likely state sequence 
in HMM A that produces O. 

3. Adjust the model parameters A to maximize P(0|A). 

The above problems are solved, respectively, by the Forward, Vitterbi and 
Baum-Welch algorithms. The effectiveness of those algorithms is based on op¬ 
timized procedure of computation, which uses a ‘trellis’: a two dimensional 
lattice structure of observations and states. This formulation is based on the 
Markov property of model evolution and reduces the complexity from exponen¬ 
tial 0(TN t ) to polynomial 0(N 2 T), where T is the number of observations 
and N the number of model states [T|. 

Depending on the formulation, there are two definitions of a Hidden Markov 
Model: Mealy and Moore. In the former, the probability of next state being 
nt+i depends both on the current state nt and the generated output symbol 
Of. In the latter, the symbol generation is independent from state switch, i.e. 
P{nt +i = S z \o t+1 = o,nt = Sj) = P[rit +1 = Si\nt = Sj). While the expressive 
power of Moore and Mealy models is the same, i.e. a process can be realized 
with Moore model if and only if it is realizable by Mealy model, the minimal 
model order for the realization is lower in Mealy models [2]. In this work we 
focus only on Mealy models. 

1.1 Related work 

In this work we follow the scheme proposed by Gudder in |3] and extend it 
in order to construct Quantum Hidden Markov Models (QHMMs). Gudder 
introduced the notions of Transition Operation Matrices and Vector States, 
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which give an elegant extension of classical stochastic matrices and probability 
distributions. These notions allow to define Markov processes that exhibit both 
classical and quantum behaviour. 

Below we review two areas of research most closely related to our work: open 
quantum walks and Hidden quantum Markov models. 

Open quantum walks In recent years a new sub-field of quantum walks 
has emerged. In series of papers H 010 n m Attal, Sabot, Sinayskiy, and 
Petruccione introduced the notion of Open Quantum Walks. Theorems for 
limit distributions of open quantum random walks were provided in [TO]. In 
m the average position and the symmetry of distribution in the SU( 2) Open 
Quantum Walk is studied. The notion of open quantum walks is generalised to 
quantum operations of any rank in m and analysed in m In first of these 
two papers the notion of mean first passage time for a generalised quantum 
walk is introduced and studied for class of walks on Apollonian networks. In 
the second paper a central limit theorem for reducible and irreducible open 
quantum walks is provided. In a recent paper m authors introduce the notion 
of hybrid quantum automaton an object similar to quantum hidden Markov 
model. They use hybrid quantum automata and derived concepts in application 
to model checking. 

Quantum hidden Markov models Hidden quantum Markov models were 
introduced in m- The construction provided there by the authors is different 
from ours. In their work the hidden quantum Markov model consists of a set of 
quantum operations associated with emission symbols. The evolution of the sys¬ 
tem is governed by the application of quantum operations on a quantum state. 
The sequence of emitted symbols defines the sequence of quantum operations 
being applied on the initial state of the hidden quantum Markov model. 

1.2 Our contribution 

In this work we propose a Quantum Hidden Markov model formulation us¬ 
ing the notions of Transition Operation Matrices. We focus on Mealy models, 
for which we derive first the Forward algorithm in general case, then the Vit- 
terbi algorithm, for models restricted to those in which sub-TOMs’ elements 
are trace-monotonicity preserving quantum operations. Subsequently, we dis¬ 
cuss the relationship between our model and model presented in [15] . The paper 
ends with the example of application proposed model. 

The paper is organised as follows: in Section [5] we collect the basic math¬ 
ematical objects and their properties, in Section [3] we define Quantum Hidden 
Markov Models and provide Forward na Viterbi algorithm for these models, in 
Section [T] we discuss the correspondences between proposed models and models 
described in m , Section [5] contains examples of application of our model, and 
finally in Section [G] we conclude. 
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2 Transition Operation Matrices 

In what follows we provide basic elements of quantum information theory and 
summarize definitions and properties of objects introduced by Gudder in [3]. 

2.1 Quantum theory 

Let T~L be a complex finite Hilbert space and C(H) be the set of linear operators 
on 7£ We also denote the set of positive operators on H. as V + (H) and the set 
of positive semi-definite operators on H as V{H). 

Definition 1 (Quantum state) A linear operator p £ V(H) is called a quan¬ 
tum state if tr p = 1. Set of quantum states is denoted by 

Definition 2 (Sub-normalised quantum state) A linear operator p £ V(TL) 
is called sub-normalised m quantum state if tr p < 1. Set of sub-normalised 
quantum states is denoted by C<(H). 

Definition 3 (Positive map) A linear map $ £ £(£(77i), £( 7 ^ 2 )) is called 
positive map if, for every p £ T(H\), $(/?) £ V^f). 

Definition 4 (Completely positive map) A linear map $ £ £(£(7h), £( 7 ^ 2 )) 
is called completely positive (CP) if for any complex Hilbert space H 3 , the map 
$ (g> 1 € C(C(Hi ® 7 ^ 3 ), £(% 0 H 3 )) is positive. 

Definition 5 (Trace preserving map) A linear map $ £ £(£(77i), £(% 2 )) 
is called trace preserving if ti(Q(p)) = tr p for every p £ £(7£). 

Definition 6 (Trace non-increasing map) A linear map <f> £ £(£(7h), £( 7 ^ 2 )) 
is called trace non-increasing if tr(4’(p)) < trp = 1 for every quantum state 
p £ h?(7ti). 

Definition 7 (Quantum operation) A linear map $ £ £(£(7ti), £( 7 ^ 2 )) is 

called a quantum operation if it is completely positive and trace non-increasing. 

Definition 8 (Quantum channel) A linear map $ £ £(£(7ti),£( 7 ^ 2 )) is 
called a quantum channel if it is completely positive and trace preserving. 

Definition 9 (Quantum measurement) By quantum measurement we call 
a mapping from a finite set 0 of measurement outcomes to subset of set of 
measurement operators p, : 0 —>• V(H) such that p(a) = 1. 

a(E© 

With each measurement p we associate a non-negative functional p : 0 —>• 

IR_(_ U {0} which maps measurement outcome a for a given positive operator p 
and measurement p to non-negative real number in the following way p(a) p = 
tr p(a)p. If tr p = 1, for given p and p the value of p can be interpreted as 
probability of obtaining measurement outcome a in quantum state p. 

If p is a sub-normalised state the trivial measurement p : a e 1 measures 
the probability p(a e ) p = tr p that the state p exists. One should note that this 
kind of measurement commutes with any other measurement and thus does not 
disturb the quantum system. 
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2.2 Transition Operation Matrices 

The core object of the Gudder’s scheme is Transition Operation Matrix (TOM) 
which generalizes the idea of stochastic matrix. 

Definition 10 (Transition Operation Matrix) Let Hi, H 2 denote two fi¬ 
nite dimensional Hilbert spaces and £2 (Hi), f2(H 2 ) denote sets of quantum states 
acting on those spaces respectively. 

A TOM is a matrix in form £ = {£ij}Yf=\, where £ij is completely positive 
map in C(C(Hi), C(H 2 )) such that for every j and p G £2(Hi) 'Yh i £ij (, p ) e 
£2(H 2 ). 

Alternatively one can say that £ = is a TOM if and only if for 

every column j JA £ij is a quantum channel (Completely Positive Trace Pre¬ 
serving map). A simple implication of this definition is that each £jj is CP-TNI 
mapping. 

Note that in this definition TOM has four parameters: 

• size of matrix “output” (number of rows) — M, 

• size of matrix “input” (number of columns) — N, 

• “input” Hilbert space - Hi, 

• “output” Hilbert space - H 2 . 

The set of TOMs we will denote as T M,N (Hi,H 2 ). 

Definition 11 (Sub Transition Operation Matrix) Let Hi, H 2 denote two 
finite dimensional Hilbert spaces, £2 (Hi) denotes set of quantum states acting 
on the first space and f2-(H 2 ) denotes set of sub-normalised quantum states 
acting on the second Hilbeid space. 

A sub-TOM is a matrix in form £ = {£ij}Yf=i, where £ij is completely posi¬ 
tive map in C(C(Hi), C(H 2 )) such that for every j and p G £2(Hi). 'f2 i £ij (p)e 

nHUi). 

The set of sub-TOMs we will denote as T< ,N (Hi, H 2 ). 

Definition 12 (Quantum Markov chain) Let a TOM £ = {£ij}ij=i be given. 
Quantum Markov chain is a finite directed graph G = (E, V ) labelled by £ij for 
e € E and by zero operator for e ^ E. 

Definition 13 (Vector state) Vector state is a column vector a = [ai, a 2 ,..., Oat] T 
such that ai G £2<(H) are sub-normalised quantum states and JA =1 Oj G fi(H). 

We will denote the set of vector states as A N (H). 

Definition 14 (Subnormalised vector state) Subnormalised vector state is 
a column vector a = [op, a 2 ,..., ajv] T such that ai G f2<(H) are sub-normalised 
quantum states and a i G fl<(H). W will denote a set of sub-normalised 

vector states as A(f(H). 


5 



Theorem 1 (Gudder [3]) Applying TOM £ G T M ’ N {TL\,TL 2 ) on a vector 
state a G A N (Hi) produces vector state /3 = £{a ) G A M {H 2 ) where a = 
[a 1 ,a 2 ,.--,a N } T , cq G 12- (Hi), where fd = [A, /3 2 , ■ ■ ■, Pm} t , A G fi-{TL 2 ), 
and £ G T M ’ N ('Hi,'H 2 ), and in the following way A = £ij( a j)- 

Theorem 2 (Gudder [3j) Product of TOM A G T M ' N ifhL\,TL 2 ) andB G V N ’ K (% 2 , TLf) 
is a TOM T M ’ K (Hi, H 3 ) 9 C = BA. 

Lemma 1 (Product of two sub-TOMs is a sub-TOM) Product of sub-TOMs 
A G rf N (ni,H 2 ) and B G T^ K (H 2 ,H 3 ) is a sub-TOM V^’ K (Hi, H 3 ) 9 C = 

BA. 

Proof 1 (Lemma |T|) According to proof of Lemma 2.2 in f3|/ ; A? = BijAij 
is a completely positive map. For every p G f2(fHi) and j we have that a = 

1 Aij(p) G Q-{H 2 ). If tr(cr) > 0 then a = <r/tr(cr) G THTL 2 ) and 

M M 

tr ( H Bi i ( f7 )) = tr ( tr ( CT ) B A (^)) 

i=1 i=1 ( 1 ) 

M v ’ 

= tr(cr)tr Aj(ct)) < 1. 

i=l 

In Ae case where tr(cr) = 0, Ae a is the zero operator and YliL 1 Aj(c) is a/so 
Ae zero operator. Thus tr AA 17 )) = 0. Hence , YltLi Ci,j{p) G F2-(H 3 ) 

andC €T^’ K (Hi,n 3 ). 

Product of (sub-)TOMs that have same dimensions is associative i.e. (£H)Q = 

£(TQ) and (AA)(a) = £(T(c$). 

3 Mealy Quantum Hidden Markov Model 

In order to explain the idea of QHMM we can form following analogy. A QHMM 
might by understood as a system consisting of a particle that has an internal 
sub-normalised quantum state p G Q<('H) and it occupies a classical state A- 
This particle hops from one classical state A into another state Sj passing 
trough a quantum operation associated with a sub-TOM element Vg k s .. With 
each transition a symbol 14 is emitted from the system. 

We will now define the classical and quantum version of the Mealy Hidden 
Markov Model. 

Definition 15 (Finite sequences) Let 

y T = VxVx...xV 
T 

defines the set of sequences of length T over alphabet V. 
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Definition 16 (Mealy Hidden Markov Model) Let S = (Si,..., Sn} and 

V = {Vi,...,V M } be a set of states and an alphabet respectively. The Mealy 
HMM is specified by a tuple A = (<S, V, II, 7r), where: 

• 7r £ [0,1]^ is a stochastic vector representing initial states, where 7q is the 
probability that the initial state is Si; 

• II is a mapping V 9 V) i —> H Vi £ R N ’ N , where H Vi is sub-stochastic matrix, 

M 

such that n s := Y n Vi £ R N ’ N is stochastic matrix and ICV is p(nt+i = 
i=1 

Sk,Ot +1 = Vi\nt = Sj), that is probability of going from state j to k while 
generating the output Vi. 

Let O = 0 \ 02 ,- ■ - Ot £ V T be a sequence of length T and P : V T —> [0,1] be 
string probabilities, defined as P(O) = p(0( 1) = Oi, 0(2) = 02 ,..., 0(T) = ot )■ 
The concatenation of string O and ot+i is denoted by Oot+i- 
It is well known that for HMMs the function P satisfies 

• Eoev-r P (°) = 1 and 

• Eot+i6V P(0°t+i) = P(0), which follows from the law of total proba¬ 
bility. 

The string probabilities generated by Mealy HMM A = (S , V, II. n) are given 
by 

N 

P(0 |A) = 5>i, 

2=1 

where on is z-th element of a = n OT II OT - 1 ... II 0l 7r. 

Definition 17 (Mealy Quantum Hidden Markov Model) Let S andV be 
a set of states and an alphabet respectively. Mealy QHMM is specified by a tuple 
A = (<S, V, V , 7 r), where: 

• 7 r £ A N (Tl) is an initial vector state; 

• V is a mapping V —> T^' N such that V s := Y p ' * € T N ' N {%,%) 

ViGV 

is a TOM, with V Vi being value of V for Vi. 

As an example we give a three-state two-symbol Mealy QHMM A = (S, V, n, 7r), 
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with 


5 

V 

n 


7 r 


■pVi 


-p V 2 


{Si,S 2 ,S 3 }, 

W,H 2 }, 

{Fi ^V v \V 2 


TTSi 
7TS 2 
.TTSs. 

pe>Vi 

f? Sl 

S3 Si 

r-p v 2 
1 Si Sl 
T > V 2 

I S, Sl 

T) *' 2 
L' S3S1 


-pVi 

GS)S 2 

-pVi 

1 S2S2 

-pl-1 

' S 3 S 2 

■p y 2 

1 Sl S 2 
'pV 2 

r S 2 S 2 

-pt2 

GS 3 S 2 


V V2 } , 


■pVi 1 
■pVi 

fy s3 

•pV 1 

' S3 S3 
-pV'2 1 

'&s 3 

-pV2 

’ S ? s 3 

T > v 2 

' S3S3J 


A graphical representation of this QHMM is presented in Fig. [T] 


^2 l^i ^ Sa |Vi 



Figure 1: Graphical representation of three-state Mealy QHMM A, whose alpha¬ 
bet consists of two symbols Vi, V 2 . The symbol should be understood 

in the following way: when QHMM is in state S 3 and is being transformed 
to state S 2 while emitting symbol V\, then the internal quantum sub-state is 
transformed by quantum operation \V_ 1 . 


Remark 1 For di mTL = 1 QHMM reduces to classical HMM. In this case 
TOMs reduce to stochastic matrices, sub-TOMs to sub-stochastic matrices, the 
vector states to probability vectors, sub-vector states to sub-normalised probabil¬ 
ity vectors. 
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3.1 Forward algorithm for Mealy QHMM 

With each Mealy QHMM we can associate a mapping 

e- v* 

where V* = Ut=iF T - Given a sequence O = (o 1 , o- 2 , ■ ■ ■, ot) and a Mealy 
QHMM A one can compute resulting sub-normalised quantum state po\\- 
Let us consider sub-normalised vector states 


a T = [q:t,i, • • •, &t,n} T G A< ('H) 


such that 

a T = V OT ...P° 2 V 0l (n), (2) 

N 

then p 0 \x := g(0) = J2 a T,i■ 

i =1 

Equation © we call the Forward algorithm for QHMMs. Note that the 
result of this algorithm is a sub-normalised quantum state po\\ G The 

sum of all those states over all possible sequences of a given length forms a 
quantum state, as formulated in the following theorem. 

Theorem 3 For any QHMM A we have po\\ G 

Ogv t 

In order to prove this theorem we will first prove the following lemma. 
Lemma 2 For any QHMM A the following holds 

V OT V OT - 1 G A n {H). 

oev T 

Proof 2 Lemma [ H We will proceed by induction. For case T = 1 we have 

Pi = '£ V°{tt) = V s (tt) G A n (H). 
oev 


For case T = n + 1 


Pt = 52 V ON+1 V ON ...V 01 Qt) = 
oev N + 1 

52 ■p°"+ i 52 ' p ° n v ° n ~ i ... v oi { 7 r) = 

o„ + lev oev w 

' p s 52 V° N V° N ~ 1 .. 
o&v N 


(3) 




By inductive hypothesis X is a TOM. V s is a TOM, therefore Pt G A N (H). 
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Proof 3 Theorem\3\ 


E p° i* = 

oev T 

N 

= E J2 a T,i 

oev T i= i 

N 

= E E^ 0 "--^ 02 ^ 1 ^ 

OGV T i =1 (4) 

N 

=£[£ •pOT p0 2 pOi 

i=l OGV T 

v -V-' 

AT 

=E[^)] i 

i=l 

Since by Lemma\^X is a TOM, therefore Y Po\\ £ 0(TL). 

oev T 

Theorem 4 Lei O = (oi, 02 ,..., ot) be a sequence of length T and Oot+ 1 be a 
concatenation of O and ot+ 1 , f/ien for any QHMM A f/ie following holds 

E POo t+ i|A = Po|a- (5) 

ot+ 1 £V 

Proof 4 Theorem [|] 

According to law of total probability for TOMs w we get 

N 

E POot +1 |A= E E^° t+1 («T,0 

ot+iGV ot+iGV i —1 

N 

=E E n ot+1 («T,o (6) 

z=l ot+iGV 
N 

= E«^ = PO|A- 
i=l 

3.2 Viterbi algorithm for Mealy QHMM 

We are given a QHMM A with set of states 5 = {Si, S 2 , ■ ■ ■, Sat} and an alphabet 
of symbols V = {Vi, V 2 ,..., Vm}- We will denote V,P := V^ s .. 

We have a sequence of length T, O = { 01 , 02 , ■■■ ,ot), of symbols from al¬ 
phabet V, Oi £ V. 

A Mealy QHMM emits symbols on transition from one state to the next. 
For our sequence O we index corresponding QHMM states by n i, i.e. no is the 
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initial state (before the emission of the first symbol), and rii,i > 1 is the state 
after emission of the symbol Oi. rii £ S. 

The goal of the algorithm is to find most likely sequence of states conditioned 
on a sequence of emitted symbols O. 

We denote the set of partial sequences of state indexes as N k = {(no, ni,... ,n k ) : 
rij £ S,j = 0,1, ... ,k}, where k <T. A set beginning with no and ending after 
k steps with Si we denote = {(no,ni,... ,n k -i,n k = Si) : rij £ S,j = 
0,1,...,A:} C N k . 

Theorem 5 Let O be a given sequence of emissions from V. Let A = (S, V, V. tt) 
be a Mealy QHMM satisfying 


'^ni,n j eS,oe0^a,peO<CH) tr a > tr P 

=> tT K unj ( a ) > ftKi.njiP) 


(7) 


i.e. all sub-TOMs elements are trace-monotonicity preserving quantum opera¬ 
tions. 

We define w £ N k l to be a sequence of k states ending with Si. A sub¬ 
normalised state associated with w and sequence O is B w £ Q<(fH) defined as 
B w = Vff k nk l 'Pn k k l\mk -2 ■ ■ ■Bn\ tno {^n 0 )- The sub-normalised state that maxi- 
mizes trace over set of all B w s with w £ N k t is 


Ak,Si = argrnax tr B w . 

{B m :weNf} 

Then the following holds 

tr A k , Si = max trp°‘ =s 

n fc_lto 

Proof 5 Let us denote 

wl Si =(n* 0 ,...,n* k _ 1 ,n%=S i )£N^ 
as the sequence of states maximizing trace of B w , so that 

tr A k , Si = tr B„ 

We now have 

tr A kt s t = max tr B w = 


u k,Si 


weN k l 

n „ maX n _ s ^Kln k . x Kt-\,n k . 2 

no,...,nk-i,rik=Oi 


■Klnoi^no) 


( 8 ) 


(9) 


( 10 ) 


(ID 


( 12 ) 


Obviously 


tr A ktSi 


trV° k 'p ° k ~ 1 

n k’ n k-i 




(13) 
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We will now prove that for = Si 


w k,Si = K,..., nl^nl) 


w, 


— ("0. ‘ ‘ > n k-1 )' 


Let us assume that it is not true. That would mean that 

w k-l,nl_ 1 ~ (‘0i • • • > *fc-2i n fc-l) / ( n 0; ■ • ■ i n k-2i n k-ll- 

Of course 

From this, and we have 

Vn k eS,y k e{l,2,...M} tT Kt,nl_S B (l*o,--Ol_ 2 ,nt_ x )) > 

that leads to 

w k,Si / 

which is a contradiction. That proves that implication ra holds. 
Then, for nk = Si 


( 14 ) 


tr A ktSi = tr B u 


— tr V k 
1 7 n ii n i -1 


(Bw * » ) — 

K,n k — 1 

= tr V° n k k=Sunki {A k -i, 


(15) 


Remark 2 It can he easily seen that Theorem [7] holds iff quantum operation 
of form c ■ $, where c € [0,1) and $ is a quantum channel (CP-TP 

map). 


From Theorem [5] we immediately derive the Viterbi algorithm for Mealy 
QHMMs conditioned with |0 that computes most likely sequence of states for 
a given sequence O. 

Initialization: 


Computation for step number k: 


n k -i&S 


: = n Si 

(16) 

n k-i(Si) = 
h,n k -i (^fe-li«fc-i) 

(17) 


(18) 


12 


Termination: 


n* T = argmaxtr Aj^. (19) 

Si&S 

The most probable state sequence is (uq, ..., n^), with resulting state being 
with probability given by tr At^- 

In case when 0 does not apply, one can resort to exhaustive search over all 
state sequences. As a result of the multitude of possible quantum operations 
the behaviour of the Quantum Hidden Markov Model can be markedly different 
than its classical counterpart. This is similar to the relation of quantum and 
classical Markov models m- 

4 Relation with model proposed by Monras et 
al. 

In [151 hidden quantum Markov model is defined, by Monras et al., as a tu¬ 
ple consisting of: a d -level quantum system with an initial state po, alpha¬ 
bet V = {K:}, a set of quantum operations (CP-TNI maps) {/C v '} such that 
Ei K Vi is a quantum channel (CP-TP map). The system evolves in discrete 
time steps and subsequently generates symbols O = (oi, 02 ,..., or) from alpha¬ 
bet V with probability P(o t ) = tr K. Vi=0t (p t ) in every time step. After gener¬ 
ation of the symbol o* the subnormalised quantum state is updated to pt = 
K, v '~ 0t {p t _i ). Moreover, K. Vi can be represented by Kraus operators {Kj*}. 
It means, that IC Vi (p) = p(A'j / ’) t and p t = Kj i=0t p t -i(Kj i=0t )\ 

where < 1. Here we omit the normalization factor, therefore 

with every sequence O a subnormalised quantum state is associated. 

In the case of Monras et al. model, the number of internal states is equal 
to dimension of quantum system. In our case, the states are divided into two 
distinct classes: ‘internal’ quantum states and ‘external’ classical states. Our 
model can be reduced to the model presented by Monras et al. by performing 
the following transformation. First, we need to extend the alphabet V with the 
symbol $. Second, we concatenate every sequence O with the symbol $. Third, 
we associate symbol $ with operation of partial trace over the classical system: 
IC $ (p) = tr u 2 P- Fourth, we express (sub-)vector states a as block diagonal 
(sub-)normalised quantum states and sub-TOMs V as quantum operations. 

According to the above, we can notice, that K. Vi corresponds to V Vi , whose 
elements are represented by Kraus operators hence V^\{p) = 

J2 J E k i ijP( E k i ijY- Let us construct the set of operators {E^\ ■} in the form 
E k\ j = E k\ j ® \k)(l\, then similarly as in [Tzf, it can be proved that 

j,k,l 

Now, consider vector state = V° T .. .V° 2 V 01 (ir) = [ou, 0 : 2 ,a./v] T with 
associated a block diagonal quantum state p a = ct; <g> |z)(z| G H(H 1 ® ^ 2 ), 
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then 


Po$ — tr 'H 2 ^2 E 

j,k,i 


Vi=OT 

k,l,j 


pVi=o 2 pVi=o\ 

^k,l,j ^k,l,j P c 




Vi=02\ 

k,l,j > 


®7 T ) t - 


( 20 ) 

Thus, our model can be expressed in the language proposed by Monras et al. 
However, formalism proposed in this paper has three notable advantages. First, 
it presents a hybrid quantum-classical model similar to the one presented in 
|14] therefore has similar field of applications. Our model intuitively generalizes 
both classical and quantum models. Second, this model allows us to propose 
a generalized version of Viterbi algorithm. Third, the use of TOM and vector 
states formalism reduces the amount of memory required to numerically simulate 
hybrid quantum-classical Markov models. 


5 Examples of application 


5.1 Example 1 

Let us consider alphabet V = {a,b,c}. We define a set of sequences O C V T 
of length T and having Oi = a for odd i, and O; £ { b , c} for even i, i.e. 
aba , abaca , acacabaca. 

Let T = 3. Our objective is to build a model able to differentiate sequences 
in O from all other sequences. In classical case, our model could be given by 
a HMM parametrized by AJ = (S , V, n, n), where 


S = {si,s 2 }, 


7T = 


0 

1 


n = 



o 1 
0 0 


,n b = 


o o 

3 0 


,n c = 


o 

i 

L2 



( 21 ) 


It’s obvious that p(aba\\\) = p{aca\X\) = whereas for other possible se¬ 
quences we get Eo6V r \oP(°l A i) = °- 

If we are interested in further differentiating aba from aca, we could either 
construct two HMMs, one for each sequence, i.e. for aba parametrized by = 
(. S , V, n, 7r), where 


n = <^ n a = 


o 1 
0 0 


,n b = 


o o 

3 0. 


( 22 ) 


and similarly for aca , or by building a three-state HMM A§ = (5, V, n, n) 

0 

S = {s 1 ,s 2 ,s 3 }, 7T = 1 

0 


H=< 

n a = 

0 1 1 
0 0 0 

,n b = 

'0 o o' 

3 0 0 

,n c = 

'0 0 o' 
0 0 0 

1 


l 

1 

o 

o 

O | 


0 0 0 


.3 0 0 

1 


( 23 ) 
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and recognize the sequences —aba from aca— based on the output of Vitterbi 
algorithm. 

We can solve the problem of discrimination by using QHMM Af = ( S , V, V , 7 r), 
with the following parameters 


S = {si,s 2 }, 


0 2 

| 0 )( 0 | 


n = {v a = 


o 4 i 4 
o 4 o 4 


,v b = 


o 4 

h*u 


o 4 

o 4 


iV c = 


04 04 

±1 4 o 4 


(24) 


where $[/(•) = U-U* is unitary channel, such that U = 


cos 77 — sin 


and CL 


t n are zero and identity operators over vector space of dimension n, respectively. 
Moreover, let fj, : {b >-> [1)(1|, c >->■ |0)(0|} be a measurement. 

Let us consider the application of quantum forward algorithm on sequence 

" 0 2 " 


aba. Initial vector state of the algorithm is Oq = 


«3 = 




u f 


| 0 )( 0 | 


, the final state is 


0 2 


. The associate sub-normalised quantum state is p a ba\xl = 


jU 10X01 U\ therefore the resulting sequence of probabilities is given by 


(trPo&a|A?M(&).trp a 6 0 |A«Mc)) = ( o’ 0 


(25) 


It is obvious that application of quantum forward algorithm on sequence aca 
gives result (0 ,\) and Eoev r \C tr Po\\\ = °- 

We have shown that it is possible to construct two-state QHMM that fulfils 
the same task as pair of two-states HMMs or three-state HMM. 


5.2 Example 2 

Let us consider language A consisting of the sequences a kl b k2 a k3 ■ ■ ■, where 
ki, & 2 , ^ 3 ) ••• are nonnegative odd integers and a,b are symbols from alphabet 
V = {«,&}• In other words language A contains these sentences in which odd 
length subsequences of letters a and b alternate. 

Classically, sequences from this language can be generated by four-state 
HMM A c = (S , V, n, 7 r) presented in Fig. 2(a)| where 


, S2 , S3, s 4 }, 


n = <^ n a = 


0100 
\ 0 0 0 
6000 

J ° ° ° 


,n b = 


0 0 0 i' 
0000 
0 0 0 i 
0010 


(26) 
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It is easy to check, that for any sequence a kl b k2 a ks ■ ■ ■ from the language 


A , probability p(a kl b k2 a ks ■ ■ ■ |A C ) is nonzero and equals p(a kl b k2 a k3 


2 )^“ ( 2 )^"” (g) - ^ ■ Moreover, if any fcj is even, thenp(a fcl 6 2 a 3 

Let us consider matrix of probabilities p(a kl b k2 a k3 ■ ■ ■ |A C ) given as 


|A C ) = 

'' |A C ) = 0. 


1 

p(a\X c ) 

P(b |A C ) 

p(aa A°) 

p(ab\X c ) 

p(ba\X c ) 

p(a\\ c ) 

p(aa A c ) 

p(ba\X c ) 

p(aaa |A C ) 

p(aba\X c ) 

p(baa\X c ) 

p(b\\ c ) 

p(ab\X c ) 

p(bb\X c ) 

p{aab\X c ) 

p(abb\X c ) 

p{bab\X c ) 

p(aa A°) 

p(aaa |A C ) 

p(baa\X c ) 

p(aaaa |A°) 

p(abaa |A C ) 

p(baaa\X c ) 

p(ab\X c ) 

p\aab A°) 

p(bab\X c ) 

p(aaab\X c ) 

p(abab\X c ) 

p(baab\X c ) 

p(ba\X c ) 

p(aba A c ) 

p(bba\X c ) 

p(aaba |A C ) 

p{abba\X c ) 

p(baba\X c ) 


(27) 


Notice, that any upper-left corner of matrix H is known as the Hankel ma¬ 
trix. Denote by Hd a upper-left d-size submatrix of matrix H. Subsequently, 


let us notice that 

1 

1 

2 

0 

1 

2 

1 

4 

0 

0 

1 

4 

0 

1 

8 

1 

4 



1 

2 

1 

2 

0 

1 

4 

1 

8 

0 

0 

1 

4 

0 

1 

8 

0 



0 

1 

4 

0 

0 

1 

4 

0 

0 

1 

8 

0 

1 

16 

1 

8 



1 

2 

1 

4 

0 

1 

4 

1 

8 

0 

0 

1 

8 

0 

1 

16 

0 



1 

4 

0 

0 

1 

8 

1 

16 

0 

0 

0 

0 

0 

0 


rank(Hn) = rank 

0 

1 

8 

0 

0 

0 

0 

0 

1 

16 

0 

1 

32 

1 

16 

= 4. (28) 


0 

1 

4 

0 

0 

1 

8 

0 

0 

1 

8 

0 

1 

16 

1 

8 



1 

4 

1 

4 

0 

1 

8 

1 

16 

0 

0 

1 

8 

0 

1 

16 

0 



0 

1 

8 

0 

0 

0 

0 

0 

1 

16 

0 

1 

32 

0 



1 

8 

0 

0 

1 

16 

1 

32 

0 

0 

0 

0 

0 

0 



1 

_4 

0 

0 

1 

8 

1 

16 

0 

0 

0 

0 

0 

0 



Since rank(idn) = 4, four-state HMM A c = (5, V, II, 7r) cannot be reduced to 
HMM with smaller number of states mm- 

The application of the QHMM for the generation of sequences from A can re¬ 
duce the number of the states to three. Let us consider QHMM A 9 = (<S, V, n, n) 
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presented in Fig. |2(b)[ with 


S {^1; ^2, ^ 3 }) 


7 r 



! = 



0 4 

$ I+X+| 

1 

0 

X 

0 


0 4 

0 4 

0 4 

1 

®H\Q\0\ 

°4 

0 0 

1 

0 0 

,v b = 

±5 0 

X 

0 4 

$ I-X-| 

$ tf|ixi| 

0 4 

1 


(29) 


where $*(•) = X ■ X* and X € {|+)<+|, |-)(-|, i?|0)(0|, H|l)(l|}. 



Figure 2: Examples of HMM (a) and QHMM (b) generating with nonzero prob¬ 
abilities sequences a kl b k2 a k3 • • •, where ki, k 2 , k 3 , • • ■ are nonnegative odd inte¬ 
gers. 


Notice that, for any sequence a kl b k2 a ks • • •, where ki,k 2 ,k 3 ,... are nonneg¬ 
ative odd integers, the final state is given as 


or 


&kik2 & 3 -" 



1 1 

1 1 


o 2 



o 2 




&kik 2 k3... 


o 2 

o 2 


1 -1 

-1 1 


Moreover, if any ki is even, then ctk 1 k 2 k 3 ... 


0 2 

0 2 

0 2 


Therefore we have shown, 


that it is possible to construct thee-state QHMM generating sequences from 
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A with the same probabilities like its classical four-state counterpart. Those 
probabilities tr p a k ib k 2 a k 3 ---\\i are obtained from trivial measurements of sub¬ 
normalised quantum states p a k lb k 2a k 3 


6 Conclusions 

We have introduced a new model of Quantum Hidden Markov Models based on 
the notions of Transition Operation Matrices and Vector States. We have shown 
that for a subclass of QHMMs and emission sequences the modified Viterbi 
algorithm can be used to calculate the most likely sequence of internal states 
that lead to a given emission sequence. Because of the fact that the structure 
of Quantum Hidden Markov Models is more complicated than their classical 
counterparts, in general case the most likely sequence of states leading to a 
given emissions sequence has to be calculated using extensive search. We have 
also proposed a formulation of the Forward algorithm that is applicable for 
general QHMMs. 

For given a sequence of symbols of length T, O = ( 01 , 02 , ..., ot), a sequence 
of states Nt = (no, n\, ...tit) and a classical Mealy HMM with parameters A, 
the joint probability distribution P(Nt , O) can be factored into 

T 

P(Nt,0) = P(tio) Y[P( n t\ot,nt-i)P(ot\nt-i). (30) 

t =1 

As in the case of classical Moore HMM [l9j , the above factorization can be con¬ 
sidered as a simple dynamic Bayesian Network. Hence, the concept of QHMM 
proposed in this manuscript gives basis to quantum generalization of dynamic 
Bayesian Networks. 

We believe that proposed model can find applications in modelling systems 
that posses both quantum and classical features. 
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